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ABSTRACT 


The  source  selection  decision  in  a  competitively 
negotiated  acquisition  is  very  difficult.  Cost  realism  is 
one  evaluation  factor  that  may  be  used  to  assist  in  making 
the  source  selection  decision.  In  this  study  the  concept 
of  cost  realism  and  feasible  methodologies  for  achieving  it 
are  investigated.  The  role  of  cost  realism  in  the  source 
selection  plan  and  the  source  selection  process  is 
identified  and  defined.  Specific  cost  realism  issues  are 
reviewed  and  discussed.  Cost  realism  utilization  in  the 
Department  of  the  Navy  is  investigated.  The  study 
concludes  that  cost  realism  provides  useful  assistance  in 
making  the  source  selection  decision. 
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INTRODUCTION 


A.  GENERAL 

The  source  selection  decision  in  a  competitively 

negotiated  acquisition  that  will  result  in  award  of  a  cost 

reimbursable  contract  is  very  difficult.  The  contracting 

officer  or  designated  source  selection  authority  must: 

. . .ensure  selection  of  the  source  whose  proposal  has  the 
highest  degree  of  realism  and  whose  performance  is 
expected  to  best  meet  stated  government  requirements. 
[Ref.  1 :p.  15-19] 

The  selection  of  the  proposal  with  the  highest  degree 
of  realism  and  performance  is  judgemental  on  the  part  of 
the  contracting  officer  or  source  selection  authority.  The 
selection  is  based  on  a  careful  evaluation  of  the  factors 
that  make  up  the  proposal.  Additionally,  the  Federal 
Acquisition  Regulation  (FAR)  identifies  other  evaluation 
factors  that  may  be  considered  which  include  cost  realism, 
technical  excellence,  management  capability,  personnel 
qualifications,  experience,  past  performance,  schedule  and 
other  relevant  factors. 

In  procuring  products  or  services,  the  preferred  method 
is  to  issue  an  Invitation  for  Bid  (IFB)  that  completely 
identifies  the  product  or  service  desired.  A  Firm  Fixed 
Price  ( FFP )  contract  award  is  then  made  to  the  offeror  of 
the  bid  that  offers  the  lowest  cost.  This  method  of 


procurement  works  best  for  products  and  services  for  which 
performance  and/or  design  specifications  are  firm  and  more 
than  one  supplier  is  willing  and  capable  of  providing  the 
product  or  service.  When  design  or  performance 
specifications  are  not  firm,  as  in  the  case  of  research  and 
development  or  engineering  services  procurements,  there  is 
a  high  degree  of  performance,  schedule  and  cost  risk. 

To  compensate  for  these  risks  in  a  fixed  price  contract 
environment,  the  contractor  will  include  costs  for 
additional  labor,  material,  overhead  and  profit.  If  the 
additional  cost  items  are  not  utilized  during  contract 
performance,  the  contractor  keeps  the  additional  costs  as 
increased  profit.  If  the  contractor  is  unable  to  deliver 
the  required  product  or  services  within  the  contract  price, 
the  contractor  will  suffer  a  loss  that  may  be  severe  enough 
to  force  the  contractor  to  default  on  the  contract.  Both 
of  these  situations,  excess  profit  or  termination  for 
default,  are  considered  unacceptable  to  the  government. 

For  procurements  with  risk  levels  that  are  considered 
unacceptable  for  fixed  price  contracts,  the  government 
assumes  the  risk  by  awarding  a  cost  reimbursable  contract. 
This  type  of  contract  requires  the  government  to  reimburse 
all  of  the  contractor's  allowable  and  allocable  costs 
incurred  during  contract  performance.  Choosing  a  contract 
source  for  a  cost  reimbursable  contract  award  is  not  as 
simple  and  easy  as  for  a  fixed  price  contract  award.  To 


make  a  contract  award  in  this  situation  based  entirely  upon 
the  lowest  proposed  cost  would  be  unrealistic,  since  the 
final  cost  of  performance  may  bear  no  resemblance  to  the 
original  proposed  cost.  Additionally,  the  award  of  a  cost 
reimbursable  contract  obligates  the  government  to  reimburse 
the  contractor's  allowable  and  allocable  costs.  Large 
increases  in  cost  during  a  contract's  performance  seriously 
undermine  both  the  Congress'  and  public's  faith  in  the  Navy 
and  significantly  impairs  the  Navy's  planning,  programming 
and  budgeting  system.  The  use  of  cost  realism  as  an 
evaluation  factor  in  the  source  selection  process  could 
provide  some  degree  of  assurance  that  proposed  costs  are 
realistic  and  defensible. 

B.  OBJECTIVE  OF  THE  RESEARCH 

The  objective  of  this  study  is  to  explore  how  the 
source  selection  process  evaluates  contractor  proposals  for 
cost  realism  and  how  it  utilizes  this  evaluation  in  source 
selection. 

C.  RESEARCH  QUESTIONS 

To  achieve  the  objective  of  the  research  the  following 
question  was  posed: 

Can  the  cost  realism  concept  be  used  to  improve  the 
source  selection  process? 

To  answer  the  basic  research  question,  the  following 
subsidiary  questions  were  asked: 

1.  What  is  cost  realism? 


2.  At  what  points  and  under  what  circumstances  may  cost 
realism  become  an  issue  in  the  source  selection 
process? 

3.  How  can  cost  realism  best  be  utilized  in  the  source 
selection  process? 

D.  RESEARCH  METHODOLOGY 

The  information  presented  in  this  research  was  obtained 
from  currently  available  procurement  related  literature  and 
personal  interviews  with  contracting  officers  and  program 
management  office  personnel.  Literature  was  obtained  from 
the  Naval  Postgraduate  School  Library,  Defense  Technical 
Information  Center,  and  Defense  Logistics  Studies 
Information  Exchange.  Applicable  regulations,  directives, 
and  instructions  that  govern  the  acquisition  process  were 
utilized.  Personnel  interviewed  during  the  research  effort 
are  identified  in  the  List  of  References. 

E.  SCOPE  AND  LIMITATIONS 

This  research  is  limited  to  cost  realism  issues  and 
policies  associated  with  major  weapons  systems  acquisitions 
in  the  Department  of  the  Navy  (DoN).  The  thrust  of  the 
research  was  on  developing  an  understanding  of  the  cost 
realism  issues  present  in  the  source  selection  process  and 
how  these  issues  are  recognized  and  dealt  with  by  the 
contracting  officer.  Source  Selection  Evaluation  Board 
( SSEB ) ,  Source  Selection  Advisory  Council  (SSAC),  and 
Source  Selection  Authority  (SSA). 


F.  ORGANIZATION  OF  THE  STUDY 


This  thesis  is  organized  to  provide  the  reader  with  an 


examination  of  the  cost  realism  concept  and  a  basic 


understanding  of  its  role  in  the  source  selection 


decision.  The  thesis  is  divided  into  the  following 


chapters : 


1.  Chapter  II  introduces  the  concept  of  cost  realism 


through  a  discussion  of  the  source  selection  environment 


and  the  source  selection  process  leading  up  to  the 


definition  of  cost  realism. 


2.  Chapter  III  reviews  and  discusses  where  and  when 


cost  realism  issues  are  specifically  addressed  during  the 


source  selection  process. 


3.  Chapter  IV  discusses  how  cost  realism  is  utilized 


in  the  Department  of  the  Navy. 


4.  Chapter  V  summarizes  the  results  of  the  research. 


provides  conclusions  and  recommendations  developed  as  a 


result  of  this  study  and  provides  recommendations  for 


further  study. 


acquisitions.  First,  the  weapon  system  begins  as  a  concept 
that  utilizes  the  latest  in  technological  advances  and  in 
some  cases,  utilizes  technology  thct  is  still  under 
development.  Second,  successful  research,  development, 
test  and  evaluation  (RDT&E)  of  a  technologically  advanced 
major  weapon  system  can  involve  many  unkown  factors  and 
their  attendant  risks.  Third,  the  government  must  carry 
most,  if  not  all,  of  the  cost  risk  associated  with  major 
weapon  system  development  through  the  award  of  a  cost 
reimbursable  contract. 

The  cost  of  a  major  weapon  system  is  often  measured  in 
hundreds  of  millions  or  billions  of  dollars.  When  there  is 
more  than  one  source  available  to  develop  and  manufacture 
the  major  weapon  system,  the  resulting  competition  may  be 
fierce  and  highly  competitive.  There  is  usually  much  more 
at  stake  than  the  profit  and  loss  statement  of  a  single 
contractor.  Thousands  of  jobs  could  be  at  risk,  not  only 
with  the  primary  contractor,  but  with  sub-contractors  and 
other  "trickle-down  economy"  business  in  the  surrounding 
communities.  Additionally,  local  governments,  special 
interest  groups  and  congressional  interests  are  always 
aroused  whenever  a  government  action  has  the  potential  for 
major  impact  on  the  economy,  environment  or  votes. 

There  is  a  large  amount  of  pressure  involved  in  the 
acquisition  of  a  major  weapon  system.  There  is  pressure  on 
the  program  manager  to  develop  an  affordable,  reliable  and 


technologically  advanced  system  on  time  and  under  budget. 
There  is  pressure  on  the  contractors  to  win  the  system 
award  competition  and  keep  profits  and  jobs  secure.  There 
is  pressure  on  the  contracting  officer  or  source  selection 
authority  to  procure  the  system  from  responsible  sources  at 
fair  and  reasonable  prices  calculated  to  result  in  the 
lowest  ultimate  cost  to  the  government. 

The  very  real  result  of  this  pressure,  when  coupled 
with  the  technical,  schedule  and  cost  risks  involved,  could 
be  the  award  and  development  of  a  major  weapon  system  based 
on  an  unrealistic  proposal. 

C.  THE  SOURCE  SELECTION  PROCESS 

Source  selection  is  the  process  wherein  the 
requirements,  facts,  recommendations  and  government 
policies  relevant  to  an  award  decision  in  a  competitive 
procurement  of  a  system  are  examined,  and  the  source 
selection  decision  is  made.  Source  selection  procedures 
are  designed  to: 

(a)  maximize  competition 

(b)  minimize  the  complexity  of  the  solicitation, 
evaluation,  and  the  selection  decision 

(c)  ensure  impartial  and  comprehensive  evaluation  of 
offerors'  proposals  and 

(d)  ensure  selection  of  the  source  whose  proposal  has  the 
highest  degree  of  realism  and  whose  performance  is 
expected  to  best  meet  stated  Government  requirements. 
[Ref.  1 :p.  15-19] 

These  procedures  are  not  only  good  business,  but  are 
also  in  keeping  with  the  responsibilities  inherent  within 
the  stewardship  of  public  monies  and  trust. 


The  determination  of  those  factors  and  attributes  which 
identify  a  responsible  source,  as  well  as  how  a  fair  and 
reasonable  price  can  be  accurately  calculated,  has  been, 
and  remains  today,  a  difficult  problem  for  DoD  to  resolve. 
Deputy  Secretary  of  Defense  Carlucci  recognized  this 
problem  when  he  reported  that: 

Some  DoD  competitively-selected  contractors  have 
performed  poorly.  In  some  instances,  source  selection 
criteria  do  not  sufficiently  take  into  account  past 
performance  or  plans  for  future  phases  of  a  program. 

Also,  the  credibility  and  realism  of  contractor  cost 
proposals  are  not  always  challenged.  [Ref.  2:p.  38] 

Failures  in  the  DoD  source  selection  process  can  easily 
result  in  unanticipated  program  cost  growth  which  can  then 
lead  to  budgetary  shortfalls  or  schedule  slippage  due  to 
inadequate  funding.  Additionally,  when  it  becomes 
necessary  to  fix  the  blame,  which  it  almost  invariably 
does,  the  resultant  finger  pointing,  accusations, 
counter-accusations,  congressional  hearings.  Inspector 
General  and  General  Accounting  Office  investigations  and 
audits  can  result  in  either  program  redirection,  reduction 
or  cancellation.  There  is  also  the  very  real  possibility 
for  additional  statutory  or  regulatory  guidance  being  added 
to  the  already  overwhelming  and  overburdened  acquisition 
system. 

The  Department  of  Defense  recognized  the  problems 
associated  with  inadequate  source  selection  decisions  when 


it  issued  Department  of  Defense  Directive  (DODD)  4105.62, 
Selection  of  Contractual  Sources  for  Major  Defense  Systems, 
establishing  detailed  guidelines  for  the  management  of  the 
source  selection  process. 

As  part  of  the  policy  which  governs  the  solicitation, 
evaluation  and  source  selection  process,  DODD  4105.62 
states : 

1.  The  prime  objectives  of  the  process  are  to  (a) 
select  the  source  whose  proposal  has  the  highest  degree 
of  realism  and  credibility  and  whose  performance  is 
expected  to  best  meet  Government  objectives  at  an 
affordable  cost;  (b)  assure  impartial,  equitable,  and 
comprehensive  evaluation  of  competitors’  proposals  and 
related  capabilities;  and  (c)  maximize  efficiency  and 
minimize  complexity  of  solicitation,  evaluation  and  the 
selection  decision. 

2.  Each  DoD  component  shall  develop,  and 
consistently  apply,  procedures  which  create  the 
environment  for  an  impartial,  balanced  and  realistic 
appraisal  of  all  proposals  submitted. 

The  prime  objectives  of  the  source  selection  process 
are  therefore  in  keeping  with  Defense  Acquisition 
Regulations  (DAR)  policy  to  procure  from  responsible 
sources  at  fair  and  reasonable  prices. 

The  source  selection  process  as  currently  established 
requires  an  evaluation  of  the  contractor's  proposal  and  the 
capability  of  the  contractor  to  successfully  accomplish  the 
program  under  consideration. 

The  primary  goal  of  evaluation  is  to  determine,  for 
each  competitor,  the  most  likely  outcome  in  terms  of 
system  characteristics,  costs  (development,  production, 
and  support),  and  schedules ....  The  evaluation  should  give 
consideration  to  each  industrial  competitor's  assessment 
of  the  technical  and  financial  risks  involved  and  the 
credibility  of  proposed  approaches  to  eliminate,  avoid  or 
minimize  those  risks....  [Ref.  3:p.  6] 


This  evaluation  process  is  conducted  in  four  steps: 


Step  1.  Separate  technical  proposals  shall  be  solicited 
and  evaluated. . . . 

Step  2.  A  cost/price  proposal  shall  then  be 
obtained. .. together  with  any  necessary  revisions  to  correct 
deficiencies  in  the  technical  proposals ...  the  competitive  range 
will  be  established. . .meaningful  discussions  will  be  held  with 
the  remaining  offerors.... 

Step  3.  Following  such  discussions,  a  common  cutoff  date 
for  the  receipt  of  final  revisions  to  technical  and  cost/price 
submittals  will  be  established ...  after  receipt  of  any  revised 
submittals,  the  proposals  shall  be  evaluated  based  upon  the 
offeror's  total  proposal  and  a  contractor  selected  for 
negotiation  of  the  contract. 

Step  4.  A  definitive  contract  will  then  be  negotiated  with 
the  selected  offeror.  [Ref.  3:p.  7] 


D.  THE  ROLES  OF  COST  AND  PRICE  IN  THE  SOURCE  SELECTION 
PROCESS 

1.  Definitions 


In  the  source  selection  process,  the  determination 
of  estimates  for  cost  and  price  for  a  weapon  system  may 
become  a  point  where  the  government  and  contractor  take 
divergent  courses.  It  becomes  necessary,  therefore,  to 
define  the  terms  frequently  used  in  developing  the 
estimates  for  cost  and  price: 

a.  "Cost  Analysis"  is  the  review  and  evaluation  of  the 
separate  cost  elements  and  proposed  profit  of  (a)  an 
offeror's  or  contractor's  cost  or  pricing  data  and 
(b)  the  judgmental  factors  applied  in  projecting  from 
the  data  to  the  estimated  costs,  in  order  to  form  an 
opinion  on  the  degree  to  which  the  proposed  costs 
represent  what  the  contract  should  cost,  assuming 
reasonable  economy  and  efficiency.  [Ref.  l:p.  15-25] 


b.  "Cost  or  Pricing  data"  means  all  facts  as  of  the  time 
of  price  agreement  that  prudent  buyers  and  sellers 
would  reasonably  expect  to  affect  price  negotiations 


significantly.  Cost  or  Pricing  data  are  factual,  not 
judgmental,  and  are  therefore  verifiable.  While  they 
do  not  indicate  the  accuracy  of  the  prospective 
contractor's  judgment  about  estimated  future  costs  or 
projections,  they  do  include  the  data  forming  the 
basis  for  that  judgment.  Cost  or  Pricing  data  are 
more  than  historical  accounting  data;  the/  are  all 
the  facts  that  can  be  reasonably  expected  to 
contribute  to  the  soundness  of  estimates  of  future 
costs  and  to  the  validity  of  determinations  of  costs 
already  incurred.  They  also  include  such  factors  as 
(a)  vendor  quotations;  (b)  nonrecurring  costs;  (c) 
information  on  changes  in  production  methods  and  in 
production  or  purchasing  volume;  (d)  data  supporting 
projections  of  business  prospects  and  objectives  and 
related  operations  costs;  (e)  unit-cost  trends  such 
as  those  associated  with  labor  efficiency;  (f) 
make-or-buy  decisions;  (g)  estimated  resources  to 
attain  business  goals;  and  (h)  information  on 
management  decisions  that  could  have  a  significant 
bearing  on  costs.  [Ref.  l:p.  15-25] 

"Price"  is  the  cost  plus  any  fee  or  profit  applicable 
to  the  contract  type.  [Ref.  l:p.  15-26] 

"Price  Analysis”  is  the  process  of  examining  and 
evaluating  a  proposed  price  without  evaluating  its 
separate  cost  elements  and  proposed  profit.  [Ref.  1: 
p.  15-26] 

"Should-Cost  Analysis"  is  a  specialized  form  of  cost 
analysis  employing  an  integrated  team  of  government 
contracting,  contract  administration,  pricing,  audit, 
and  engineering  representatives.  It  differs  from 
regular  cost  analysis  in  its  depth,  in  the  fact  that 
it  is  conducted  at  the  contractor's  plant,  and  in  the 
extent  to  which  the  Government  identifies  and 
challenges  inefficiencies  in  the  contractor's 
management  and  operations  rather  than  merely 
challenging  certain  proposed  costs.  The  purpose  of 
should-cost  analysis  is  to  (1)  identify  uneconomical 
or  inefficient  practices  in  the  contractor's 
management  and  operations,  (2)  quantify  their  impact 
on  cost  in  order  to  develop  a  realistic  price 
objective  for  negotiation,  and  (3)  lead  to  both  short 
and  long-range  improvements  in  the  contractor ' s 
economy  and  efficiency.  [Ref.  l:p.  15-39] 

"Proposal  Evaluation"  is  an  assessment  of  both  the 
proposal  and  the  offeror's  ability  (as  conveyed  by 
the  proposal)  to  successfully  accomplish  the 
prospective  contract.  [Ref.  l:p.  15-21] 


2 .  The  Roles 


Before  a  weapon  system  can  reach  the  source 
selection  stage,  the  estimated  cost  of  the  weapon  system 
has  played  a  key  role  in  the  planning,  programming  and 
budgeting  process.  There  is  the  strong  possibility  that 
Congressional  authorization  and  appropriation  for  the 
weapon  system  was  based  on  the  estimated  cost  as  much  as  it 
was  based  on  the  need  for  the  systems'  capabilities. 

The  program  manager  must  ensure  that  only  realistic 
cost  and  budget  information  is  provided  and  that  such 
information  is  accurate  and  complete.  Cost  estimates  must 
include  the  full  anticipated  development,  production  and 
operational  costs  associated  with  the  program,  even  though 
this  task  is  especially  difficult  at  program  initiation. 

The  estimates  support  budgeting  and  early  funding  for 
testing,  budgeting  for  most  likely  costs,  budgeting  for 
technical  and  scheduling  risks,  capitalization  of 
production  and  specialized  operational  facilities  and 
independent  cost  analysis.  [Ref.  4:p.  1-9] 

FAR  requires  price  or  cost  to  the  government  be 
included  as  an  evaluation  factor  in  every  source 
selection.  Other  evaluation  factors  previously  identified 
that  may  apply  to  a  particular  acquisition  are  cost 
realism,  technical  excellence,  management  capability, 
personnel  qualifications,  experience,  past  performance. 
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schedule  and  other  relevant  factors.  The  estimated  cost  of 
the  contractor's  proposal  will  certainly  be  a  factor  in  the 
evaluation  process.  Cost  is  a  driving  factor  throughout 
the  system  acquisition  process,  but  it  must  be  put  in 
perspective . 

In  the  development  stage,  the  prime  concern  is  to 
find  and  engage  contractors  who  have  the  conceptual 
ideas,  manpower,  management  expertise,  facilities,  and 
the  demonstrated  experience  to  develop  a  system  capable 
of  meeting  the  mission  need.  Cost  estimates  in  the 
earlier  stages  of  the  acquisition  process  are  far  from 
precise,  and  independent  estimates  of  development  and 
Life  Cycle  Costs  (LCC)  by  an  inhouse  activity  are  needed 
to  establish  a  baseline  against  which  to  evaluate  the 
validity  of  contractor  cost  estimates.  With  due  regard 
for  its  significance,  cost  is  a  controllable  element 
which  can  be  managed  through  carefully  drawn,  properly 
executed  contracts  and  through  liberal  use  of  competition 
throughout  the  acquisition  process.  The  cost  to  develop 
a  system  is  only  one  part,  albeit  an  important  one,  of 
the  system  LCC.  Costs  incurred  in  development  can  return 
large  dividends  in  the  fora  of  lower  production  and 
maintenance  costs  as  well  as  in  improved  performance. 
[Ref.  4:p.  4-40] 

While  the  lowest  price  or  lowest  total  cost  to  the 
government  is  properly  the  deciding  factor  in  many  source 
selections,  in  certain  acquisitions  the  government  may 
select  the  source  whose  proposal  offers  the  greatest  value 
to  the  government  in  terms  of  performance  and  other 
factors . 

In  awarding  a  cost-reimbursement  contract,  the  cost 
proposal  should  not  be  controlling,  since  advance 
estimates  of  cost  may  not  be  valid  indicators  for  final 
actual  costs.  There  is  no  requirement  that 
cost-reimbursement  contracts  be  awarded  on  the  basis  of 
lowest  proposed  cost,  lowest  proposed  fee  or  the  lowest 
total  proposed  cost  plus  fee.  The  award  of 
cost-reimbursement  contracts  primarily  on  the  basis  of 
estimated  costs  may  encourage  the  submission  of 


unrealistically  low  estimates  and  increase  the  likelihood 
of  cost  overruns.  [Ref.  l:p.  15-20] 

E.  THE  COST  REALISM  CONCEPT 

1 .  General 

Cost  is  a  key  factor  in  the  source  selection 
process.  It  may  not  be  the  most  important  factor,  but  its 
importance  cannot  be  overlooked.  The  determination  of  a 
weapon  systems'  estimated  cost  through  cost  analysis  is  a 
judgemental  decision.  The  decision  may  be  influenced  by 
optimism  or  pessimism  toward  any  one  or  more  of  the  factors 
which  go  into  making  up  the  total  estimated  cost. 
Traditionally,  contractors  have  the  tendency  to  lean  toward 
optimism  in  cost  estimation  and  the  government  has  the 
tendency  to  lean  toward  pessimism.  However,  this  is  not 
always  the  rule,  and  situations  do  occur  where  the  opposite 
may  happen.  Therefore,  the  methodology  by  which  a 
contractor's  cost  proposal  is  evaluated  to  determine 
whether  it  is  reasonable,  realistic,  credible  and 
defensible  becomes  an  important  issue. 

2 .  Cost  Realism  Defined 

Cost  realism  may  be  defined  as  the  methodology  by 
which  a  contractor's  cost  proposal  is  evaluated  during  the 
source  selection  process  to  determine  with  some  degree  of 
accuracy  and  fairness  whether  or  not  it  is  a  realistic 
estimate  of  what  the  actual  costs  are  expected  to  be. 


assuming  no  change  in  the  government ' s  requirements  or  the 
contractor's  technical  approach. 

3 .  Considerations 

FAR  requires  price  or  cost  be  included  as  an 
evaluation  factor  in  every  source  selection,  but  an 
optional  cost  realism  evaluation  may  be  made  at  the 
discretion  of  agency  acquisition  officials.  There  is  no 
definitive  guidance  in  statutes  or  regulations  for  the 
conduct  of  a  cost  realism  evaluation;  however,  there  are  a 
variety  of  cost  analysis  elements  and  techniques  that  may 
be  used  to  determine  the  realism  of  proposed  costs.  When 
cost  realism  is  included  as  an  evaluation  factor,  the 
informed  judgement  rule  requires  the  contracting  officer  or 
source  selection  authority  make  an  analysis  of  proposed 
costs  sufficient  to  make  a  supportable  judgment  that  the 
costs  proposed  are  realistic. 

F .  SUMMARY 

This  chapter  provided  a  broad  overview  of  the  source 
selection  environment  and  process  for  major  weapon  system 
acquisition.  The  complexity  and  cost  of  a  major  weapon 
system  and  the  resultant  economic  impact  on  competition 
winners  and  losers  makes  the  source  selection  decision  very 
important . 

The  evaluation  of  a  proposal  to  determine  its  degree  of 
realism  requires  a  thorough  analysis  of  every  element  which 
makes  up  the  proposal.  Cost  is  a  required  evaluation 


factor  and  may  become  the  controlling  factor;  therefore,  a 
determination  of  the  realism  of  proposed  costs  becomes  a 
required  evaluation  factor. 

By  this  analogy,  an  evaluation  of  cost  realism  becomes 
required;  however,  its  methodology  is  not  defined  in 
statute  or  regulation.  This  chapter  presented  a  definition 
of  cost  realism  as  an  evaluation  factor  in  the  source 
selection  decision  process. 


I .  COST  REALISM  ISSUES  DURING  SOURCE  SELECTION 

A.  INTRODUCTION 
It  is  the  purpose  of  this  chapter  to  review  and  discuss 

where  and  when  cost  realism  issues  are  specifically 
addressed  during  the  source  selection  process.  For 
purposes  of  discussion,  the  chapter  divides  the  source 
selection  process  into  three  parts.  Part  1  encompasses  all 
steps  leading  up  to  the  release  of  the  solicitation.  Part  2 
covers  proposal  evaluation.  Part  3  addresses  the  source 
selection  decision.  Additionally,  this  chapter  briefly 
addresses  the  issue  of  cost  realism  based  protests. 

B.  PLANNING  FOR  COST  REALISM 

1 .  The  Source  Selection  Plan  Groundwork 

When  an  acquisition  objective  has  been  established, 
an  acquisition  plan  is  prepared  that  addresses  the 
acquisition  process  which  will  be  followed  to  achieve  the 
desired  objective. 

A  key  part  of  the  acquisition  planning  process  is 
the  source  selection  plan.  The  source  selection  plan,  as  a 
minimum,  includes: 

( 1 )  A  description  of  the  organization  structure 

(2)  Proposed  presolicitation  activities 

(3)  A  summary  of  the  acquisition  strategy 

(4)  A  statement  of  the  proposed  evaluation  factors  and 
their  relative  importance 

(5)  A  description  of  the  evaluation  process,  methodology 


and  techniques  to  be  used  and 

(6)  A  schedule  of  significant  milestones.  [Ref.  l:p.  15-23] 
2 .  The  Solicitation 

The  solicitation  is  the  method  by  which  the 
government's  requirements  are  conveyed  to  potential 
contractors  either  as  a  Request  for  Proposal  ( RFP )  or  as  a 
Request  for  Quotation  ( RFQ ) . 

Solicitations  shall  contain  the  information  necessary  to 

enable  prospective  contractors  to  prepare  proposals.... 

[Ref.  1 :p.  15-10] 

The  solicitation  is  prepared  using  the  uniform 
contract  format  which  facilitates  preparation  of  both  the 
solicitation  and  the  contract.  The  principal  difference 
between  the  solicitation  and  the  contract  is  the  inclusion 
of  Part  IV,  Representations  and  Instructions,  in  the 
Solicitation.  Part  IV  contains  section  K,  Representations , 


Certifications  and  Other  Statements  of  Offerors  or  Quoters, 


Section  L,  Instructions,  Conditions  and  Notices  to  Offerors 


or  Quoters  and  Section  M,  Evaluation  Factors  for  Award 


is  in  Part  IV,  Sections  L  and  M,  that  cost  realism  issues 
are  first  addressed. 

Section  L  contains: 

...solicitation  provisions  and  other  information  and 
instructions  not  required  elsewhere  to  guide  offerors  or 
quoters  in  preparing  proposals  or  quotations.  [Ref.  l:p. 
15-12] 

It  is  in  this  section  that  detailed  guidance  is 
provided  for  the  submission  of  technical  and  cost  or 


pricing  data.  The  level  of  detail  required  is  that  which 


is  considered  necessary  for  complete  technical  and  cost 
analysis  of  the  proposal.  This  routinely  requires 
submission  of  technical  and  cost  data  in  Work  Breakdown 
Structure  (WBS)  format.  In  this  way,  cost  and  technical 
comparisons  can  be  made  between  different  offerors  for  the 
same  WBS  elements,  and  greater  cost  visiblity  can  be 
provided  where  considered  necessary. 

Section  M,  of  the  solicitation: 

...shall  clearly  state  the  evaluation  factors,  including 
price  or  cost  and  any  significant  subfactors,  that  will 
be  considered  in  making  the  source  selection  and  their 
relative  importance.  Numerical  weights,  which  may  be 
employed  in  the  evaluation  of  proposals,  need  not  be 
disclosed  in  solicitation.  [Ref.  l:p.  15-20] 

However,  it  is  necessary  to: 

. . . state  the  relative  importance  the  government  places  on 
those  evalutation  factors  and  subfactors.  [Ref.  l:p. 
15-12] 

This  is  performed  by  identifying  minimum 
requirements  that  apply  to  particular  evaluation  factors 
and  significant  subfactors. 


C.  THE  PROPOSAL  EVALUATION 

Proposal  evaluation  is  an  assessment  of  both  the 
proposal  and  offeror's  ability  (as  conveyed  by  the 
proposal)  to  successfully  accomplish  the  prospective 
contract.  An  agency  shall  evaluate  competitive  proposals 
solely  on  the  factors  specified  in  the  solicitation. 

[Ref.  1 : p .  15-21] 

The  proposal  evaluation  factors  normally  specified 
in  the  solicitation  are  technical,  cost  and  management 
capability.  Other  factors  included  for  evaluation 


purposes  may  include  personnel  qualifications,  past 
performance,  reliability  and  maintainability  and  schedule. 
For  cost  realism  purposes,  the  primary  evaluation  factors 
which  require  careful  and  complete  analysis  are  technical 
and  cost. 

1 .  Technical  Analysis 

The  purpose  of  the  technical  analysis  is  to  assess 
the  offeror's  ability  to  accomplish  the  minimum  technical 
requirements.  The  analysis  looks  at  the  types  and 
quantities  of  labor  proposed,  bills  of  material,  proposed 
sub-contracts,  proposed  technical  approach,  assumptions  of 
technical  risk  and  other  pertinent  technical  elements. 

Based  on  the  results  of  the  technical  analysis,  a  summary, 
matrix  or  quantitative  ranking  of  each  technical  proposal 
can  be  made  as  specified  in  the  source  selection  plan. 

2 .  Cost  Analysis 

Methodologies  for  conducting  cost  analysis  may 

differ  depending  upon  what  is  being  procured  and  who  is 

conducting  the  cost  analysis.  However,  the  basic  goal  of 

cost  analysis  remains  the  formation  of: 

. .an  opinion  on  the  degree  to  which  the  proposed  costs 
represent  what  the  contract  should  cost,  assuming 
reasonable  economy  and  efficiency.  [Ref.  l:p.  15-25] 

The  government's  opinion  on  costs  usually  takes  the 
form  of  a  government  estimate  of  the  most  likely  cost  which 
will  be  incurred  by  the  offeror  in  fulfilling  the  terms  and 
conditions  of  the  contract. 
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While  conducting  technical  and  cost  analyses, 
questions  invariably  arise  with  regard  to  the  technical  or 
cost  elements  being  proposed.  These  questions  are  provided 
to  the  offeror  who  responds  with  the  necessary 
substantiating  documentation.  The  government  must  exercise 
great  care  in  pre-award  discussions  with  offerors  to  avoid 
providing  technical  or  cost  information  about  competing 
proposals  or  appearing  to  direct  technical  or  cost  changes 
in  the  proposal.  Overt  technical  and  cost  leveling  is  a 
prohibited  practice,  but  pre-award  discussions  frequently 
have  the  effect  of  seeing  differing  proposals  move  closer 
together  in  certain  technical  or  cost  areas.  This  can  have 
the  effect  of  bringing  the  higher  weighted  technical  scores 
of  competent  offerors  closer  together  and  cause  the  lower 
weighted  cost  score  to  become  more  important  than 
originally  intended. 

4 .  Cost  Realism  Scoring 

After  the  technical  and  cost  analysis  have  been 
completed,  the  proposal  is  evaluated  and  scored  for  cost 
realism  in  accordance  with  the  cost  realism  scoring  plan 
established  as  a  part  of  the  source  selection  plan.  There 
are  differing  methods  for  scoring  cost  realism,  with  no  one 
method  being  clearly  superior.  The  one  similarity  all 
methods  seem  to  share  is  their  use  of  the  government  cost 


estimate  developed  during  cost  analysis  as  the  baseline 
against  which  the  proposal  is  evaluated  and  scored. 

D.  SOURCE  SELECTION 

The  source  selection  decision  is  made  by  the  source 
selection  authority  using  the  evaluation  process, 
methodology  and  techniques  established  in  the  source 
selection  plan. 

The  Source  Selection  Authority  shall  consider  any 
rankings  and  ratings,  and,  if  requested,  any 
recommendations  prepared  by  evaluation  and  advisory 
groups.  [Ref.  l:p.  15-23] 

In  practice,  the  evaluation  of  a  proposal  is  normally 
performed  by  the  Source  Selection  Evaluation  Board  (SSEB) 
in  accordance  with  the  evaluation  factors  and  process 
established  in  the  solicitation  and  source  selection  plan. 
It  is  the  SSEB  that  evaluates  and  scores  proposals  for  cost 
realism. 

The  results  of  the  proposal  evaluations  by  the  SSEB  are 
then  provided  to  the  Source  Selection  Advisory  Council 
( SSAC )  for  review.  The  SSAC  will  review  the  evaluation 
efforts  of  the  SSEB  and  request  clarification  or  further 
evaluation  as  required.  When  all  evaluations  have  been 
satisfactorily  completed,  the  SSAC  will  compare  the 
evaluation  results  between  the  competing  proposals.  When 
the  SSAC  is  finished  comparing  evaluation  results,  a 
recommendation  of  a  source  for  selection  is  prepared  with 
supporting  documentation  for  forwarding  to  the  SSA. 


The  supporting  documentation  prepared  for  the  selection 
decision  shall  show  the  relative  differences  among 
proposals  and  their  strengths,  weaknesses  and  risks  in 
terms  of  the  evaluation  factors.  The  supporting 
documentation  shall  include  the  basis  and  reason  for  the 
decision.  [Ref.  l:p.  15-23] 

The  SSA  will  review  the  SSEB  proposal  evaluations  and 
the  recommendations  of  the  SSAC  for  completeness.  After 
any  additional  questions  have  been  answered  or 
clarifications  received,  the  SSA  will  make  the  source 
selection  decision. 


E.  SOURCE  SELECTION  BID  PROTEST  FOR  INADEQUATE  COST 

REALISM 

When  the  source  selection  decision  has  been  made,  the 
government  has  the  responsibility  to  promptly  notify  each 
offeror  whose  proposal  was  not  selected  for  award. 

The  notification  shall  include- 

(i)  The  number  of  offerors  solicited 

(ii)  The  number  of  proposal  received 

(iii)  The  name  and  address  of  each  offeror  receiving  an 
award 

(iv)  The  items,  quantities  and  unit  prices  of  each 
award  (if  the  number  of  items  or  other  factors 
makes  listing  unit  prices  impracticable,  only  the 
total  contract  price  need  be  furnished)  and 

(v)  in  general  terms,  the  reason  the  offeror's  proposal 
was  not  accepted,  unless  the  price  information  in 
(iv)  above  readily  reveals  the  reason. [Ref . 1 :p. 
15-42] 

At  this  point,  unsuccessful  offerors  have  the  right  to 
protest  the  proposed  contract  award  by  filing  a  protest 
with  the  contracting  officer  or  the  Comptroller  General.  A 
belief  that  the  government  conducted  inadequate  cost 


realism  analysis  is  one  reason  of  many  possible  reasons  for 
filing  such  a  protest. 

Decisions  by  the  Comptroller  General  in  cost  realism 
based  bid  protests  have  normally  hinged  on  the  informed 
judgement  rule.  The  informed  judgement  rule  is  a 
requirement  for  the  contracting  officer  to  make  an  analysis 
of  proposed  costs  that  is  sufficient  for  making  a 
supportable  judgement  that  the  proposed  costs  are 
realistic . 

If  cost  and  technical  analyses  have  been  performed  to 
the  degree  that  a  source  selection  decision  cs.i  be  made 
that  is  both  reasonable  and  consistent  with  the  evaluation 
factors  contained  in  the  solicitation,  then  the  Comptroller 
General  will  most  likely  consider  the  decision  an  informed 
judgment . 

Three  points  should  be  kept  in  mind.  First,  the 
informed  judgment  rule  is  a  flexible  not  a  fixed 
standard.  What  constitutes  inadequate  cost  realism 
analysis  in  one  set  of  circumstances  may  be  adequate  cost 
analysis  in  another ....  Second,  the  procuring  agency's 
judgment  as  to  the  methods  used  in  developing  the 
government's  cost  estimate  and  the  conclusions 
reached. . .are  entitled  to  great  weight  since  the 
procurement  agencies  are  in  the  best  position  to 
determine  the  realism  of  costs ... .Third,  a  cost  realism 
determination  will  be  considered  reasonable  if  it  is 
based  on  an  informed  judgment  that  the  proposed  costs  are 
realistic. . . .An  informed  judgment  results  from 
consideration  of  a  number  of  different  factors,  i.e. 
independent  audits,  government's  statistics  or  similar 
contracts  for  similar  services.  By  considering  these  and 
other  factors,  the  Contracting  Officer  may  develop  and 
document  the  rational  basis  necessary  for  a  cost  realism 
determination  that  satisfies  the  informed  judgment  rule. 
[Ref.  5:p.  9] 


F. 


SUMMARY 


This  chapter  discussed  at  when  and  where  cost  realism 
issues  must  be  considered  during  the  source  selection 
process.  First,  planning  for  cost  realism  occurs  when  the 
source  selection  plan  is  prepared  by  determining  those 
proposal  evaluation  factors  which  will  be  used.  Second, 
the  realism  of  proposed  costs  and  technical  performance  are 
considered  during  the  proposal  evaluation  process,  whether 
or  not  a  formal  cost  realism  analysis  takes  place. 

Finally,  the  source  selection  decision  is  made  based  on  a 
determination  that  the  selected  source  is  best  able  to  meet 
the  government's  solicitation  requirements ,  cost,  technical 
and  other  factors  considered.  The  possibililty  that  a  bid 
protest  will  be  filed  based  on  inadequate  cost  realism 
analysis  always  exists.  However,  the  likelihood  of  an 
adverse  Comptroller  General  decision  is  reduced  as  long  as 
the  source  selection  decision  is  a  supportable  judgment 
based  on  an  adequate  cost  realism  analysis. 


IV. 


COST  REALISM  UTILIZATION  IN  THE  DEPARTMENT  OF  THE  NAVY 
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A.  INTRODUCTION 

This  chapter  reviews  how  cost  realism  is  utilized  in 
the  Department  of  the  Navy.  The  chapter  begins  with  a 
discussion  of  cost  realism  requirements  and  guidance  as 
established  by  the  Office  of  the  Assistant  Secretary  of  the 
Navy,  Shipbuilding  and  Logistics  (ASN  S&L )  .  The  chapter 
then  introduces  the  cost  realism  practices  of  the  Navy's 
three  major  system  commands:  Naval  Sea  Systems  Command 
(NAVSEA),  Naval  Air  Systems  Command  (NAVAIR)  and  the  Space 
and  Naval  Warfare  Systems  Command  (SPAWAR). 

B,  ASN  (S&L)  COST  REALISM  UTILIZATION 

There  was  no  formally  established  Navy-wide  cost 
realism  guidance  in  effect  at  the  time  of  this  study.  The 
closest  thing  to  Navy-wide  guidance  available  is  the  Cost 
Realism  Handbook  for  Assuring  More  Realistic  Contractor 
Cost  Proposals  promulgated  by  the  Navy  Office  for 


rv 


Acquisition  Research: 

The  handbook  is  about  source  selection  cost  evaluation 
for  competitive,  cost  reimbursable  contracts  with 
particular  emphasis  on  assuring  more  realistic  contractor 
cost  proposals.  It  is  intended  as  a  practical  guide  and 
reference  for  program  management  personnel  and  source 
selection  personnel. 

The  handbook  discusses  the  source  selection  process  and 
the  necessity  for  adequate  cost  analysis  to  assure  the 
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realism  of  contractor  cost  proposals.  Techniques  and 
procedures  are  presented  for  conducting  cost  realism 
analysis,  but  they  are  advisory  only  and  not  mandatory  for 
source  selection  decisions. 

The  handbook  goes  into  great  detail  on  some  of  the 
different  methods  available  for  scoring  cost  realism.  The 
thrust  of  the  handbook's  cost  realism  scoring  methodology 
is  the  development  of  an  unbiased  scoring  model  against 
which  all  cost  proposals  can  be  compared  with  a  clear 
winner  emerging.  Where  the  handbook  appears  to  be 
incomplete  is  in  its  treatment  of  contractor  cost  proposals 
in  comparison  to  the  government's  cost  estimate.  The 
difference  between  the  contractor's  cost  proposal  and  the 
government's  cost  estimate  determines  the  cost  realism 
score  with  no  allowance  noted  for  positive  or  negative 
scores . 

As  an  example  of  the  handbook's  proposed  scoring 
method,  a  contractor  proposal  that  is  within  plus  or  minus 
five  percent  of  the  government  estimate  could  be  awarded  a 
maximum  score.  For  a  $20  million  government  estimate,  this 
would  mean  the  contractor's  proposal  can  be  between  $19  and 
$21  million.  This  presents  a  problem  of  contract  award. 

If  the  government  awards  the  contract  for  any  amount  less 
than  the  $20  million  government  estimate,  and  the  estimate 
was  accurate,  then  there  will  most  likely  be  a  cost  growth 
of  $1  million  in  contract  completion  costs.  If  the 


contract  is  awarded  for  an  amount  greater  than  the 
government  estimate,  two  questions  arise.  First,  how  can  a 
fair  and  reasonable  price  determination  be  made  at  an 
amount  higher  than  the  government  estimate?  Second, 
assuming  the  contractor  manages  to  the  contract  price,  does 
the  amount  of  the  contract  which  exceeds  the  government 
estimate  constitute  additional  profit  and  is  this  a 
violation  of  existing  profit  policy?  [Ref.  6] 

C.  NAVSEA  COST  REALISM  UTILIZATION 
1 .  Introduction 

NAVSEA  uses  cost  realism  analysis  to  assist  the 
source  selection  decison  making  process  in  awarding  cost 
reimbursable  ship  repair  overhaul  contracts.  [Ref.  7] 

NAVSEA  obtains  ship  repair  overhauls  through  what 
could  best  be  described  as  a  two  step  process.  In  the 
first  step,  potential  offerors  propose  against  a  notional 
specification  work  package  that  represents  NAVSEA 's  best 
estimate  of  the  work  effort  which  will  be  required  during 
the  ship  repair  overhaul.  Contractor  proposals  are 
evaluated  and  scored  in  accordance  with  the  evaluation 
factors  contained  in  the  RFP .  Included  among  these 
evaluation  factors  are  factors  for  cost  realism  and  cost  to 
government.  After  all  proposals  have  been  evaluated  and 
scored,  the  winning  offeror  is  chosen  to  receive  the  ship 
repair  overhaul  award.  At  this  point,  step  two  of  the 


process  begins  with  finalization  of  the  ship  repair 
specification  package  and  the  commencement  of  negotiations 
(sole-source)  for  the  terms,  conditions  and  price  for  the 
actual  ship  repair  overhaul  contract.  [Ref.  7] 

2 .  Cost  Realism  Analyses  Performed 

NAVSEA's  evaluation  of  a  proposal  begins  with  the 
selection  of  a  sample  of  work  items  from  the  notional 
specification  package  for  in  depth  evaluation.  This 
selection  of  sample  work  items  usually  represents  that  ten 
percent  of  the  work  package  items  that  have  the  highest 
value.  [Ref.  7] 

Evaluation  of  the  selected  work  items  is  based  upon 
the  evaluator's  review  of  the  contractor's  back-up  data 
used  to  support  the  proposed  material  and  manhours  of  labor 
to  complete  the  work  item.  If  the  backup  data  fully 
supports  the  contractor's  proposed  material  and  labor 
figures  in  both  quantity  and  quality,  no  adjustment  to  the 
proposal  is  necessary.  If  the  backup  data  does  not 
adequately  support  the  contractor's  proposed  material  or 
labor  figures,  the  proposal  is  adjusted  where  required. 

The  necessity  for  adjustment  and  the  level  of  adjustment 
taken  is  a  subjective  determination  made  by  the  NAVSEA 
evaluator ( s ) .  [Ref.  7] 

Evaluated  and  adjusted,  if  necessary,  work  item 
cost  estimates  are  compared  to  the  contractor's  proposed 
work  item  estimate,  and  any  difference  becomes  the 


adjustment  factor  for  that  work  item.  The  adjustment 
factors  for  all  work  items  evaluated  are  then  accumulated, 
and  a  cumulative  adjustment  factor  is  determined.  The 
cumulative  adjustment  factor  is  then  applied  to  the 
contractor's  proposal  resulting  in  an  adjusted  proposal, 
which  also  becomes  the  cost  to  government  estimate.  The 
percent  difference  between  the  original  proposal  and  the 
adjusted  proposal  is  used  to  score  cost  realism.  [Ref.  7] 
Cost  to  government  scoring  is  based  upon  a  ranking 
of  all  adjusted  proposals,  from  lowest  to  highest.  The 
proposal  resulting  in  the  lowest  estimated  cost  to 
government  receives  the  highest  score  with  subsequent 
scores  decreasing  as  cost  to  government  increases.  [Ref.  7] 
Scores  for  cost  realism  and  cost  to  government  are 
weighted  and  combined  with  the  weighted  scores  for  other 
evaluation  factors.  The  final  scores  for  all  offerors  are 
then  compared,  and  the  highest  scoring  offeror  is  selected 
for  contract  award.  [Ref. 7] 

D.  NAVAIR  COST  REALISM  UTILIZATION 
1 .  Introduction 

NAVAIR  uses  cost  realism  analysis  to  assist  the 
source  selection  decision  making  process  in  awarding  cost 
reimbursable  contracts.  [Ref.  8] 


2. 


Cost  Realism  Analysis  Performed 


NAVAIR's  evaluation  of  contractor  proposals  results 
in  a  thorough,  critical  review  of  every  proposal  received 
in  response  to  an  RFP.  [Ref.  8] 

NAVAIR  evaluators  review  each  element  of  the  WBS 
for  completeness,  adequacy  and  supportability .  The 
determination  of  a  contractor's  capability  to  perform  the 
effort  contained  in  a  proposal  is  made  based  upon  a  review 
of  the  proposal,  prior  performance  of  the  contractor  on 
similar  efforts  and,  in  some  cases,  prior  performance  of 
other  contractors  on  similar  efforts.  The  objective  of 
NAVAIR's  critical  review  is  the  development  of  an 
independent  estimate  of  what  the  product  will  most  probably 
cost  the  government  if  manufactured  by  the  contractor. 

This  most  probable  cost  becomes  the  government  estimate  and 
when  compared  to  the  contractor's  proposal  forms  the  basis 
for  a  determination  regarding  the  cost  realism  of  the 
contractor's  proposal.  [Ref.  8] 

The  information  developed  regarding  cost  realism  of 
the  contractor's  proposal  is  provided  to  the  SSAC  along 
with  all  other  data  generated  during  the  proposal 
evaluation  process.  The  SSAC,  after  considering  evaluation 
reports  on  all  offerors,  makes  an  award  recommendation  to 
the  SSA .  [Ref.  8] 
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E. 


SPAWAR  COST  REALISM  UTILIZATION 


1.  Introduction 


SPAWAR  uses  cost  realism  analysis  to  assist  the 
source  selection  decision  making  process  in  awarding  cost 
reimbursable  contracts.  [Ref.  9] 

2 .  Cost  Realism  Analysis  Performed 

Among  the  Navy's  three  systems  commands,  SPAWAR 
alone  has  developed  detailed  procedures  for  the  conduct  of 
cost  realism  analysis  on  contractor  proposals. 

The  procedure  begins  with  the  decision  to  include  a 
requirement  for  cost  realism  analysis  and  scoring  in  the 
source  selection  plan.  [Ref.  9]  Contractors  are  advised  in 
the  RFP  that  cost  realism  analysis  will  be  performed 
through  the  inclusion  of  the  clause: 

REALISM  OF  COST  OR  PRICE  PROPOSALS  (M-74  MAY) 

An  offeror's  proposal  is  presumed  to  represent  his  best 
efforts  to  respond  to  the  solicitation.  Any 
inconsistency,  whether  real  or  apparent,  between  promised 
performance,  and  cost  or  price,  should  be  explained  in 
the  proposal.  For  example,  if  the  intended  use  of  new 
and  innovative  production  techniques  is  the  basis  for  an 
abnormally  low  estimate,  the  nature  of  these  techniques 
and  their  impact  on  cost  or  price  should  be  explained; 
or,  if  a  corporate  policy  decision  has  been  made  to 
absorb  a  portion  of  the  estimated  cost,  that  should  be 
stated  in  the  proposal.  Any  significant  inconsistency, 
if  unexplained,  raises  a  fundamental  issue  of  the 
offeror's  understanding  of  the  nature  and  scope  of  the 
work  required  and  of  his  financial  ability  to  perform  the 
contract,  and  may  be  grounds  for  rejection  of  the 
proposal.  The  burden  of  proof  as  to  cost  credibility 
rests  with  the  offeror. 

Upon  receipt  of  a  contractor's  proposal,  a  careful 
and  complete  evaluation  of  the  proposal  is  made  by  the 


SPAWAR  evaluation  team.  Each  element  of  the  WBS  is 
examined  for  completeness,  adequacy  and  supportability . 
Comparisons  are  made  between  the  contractor's  proposed 
costs  for  material,  labor,  overhead  and  known  costs  for 
similar  efforts  by  this  contractor,  or  other  contractors  on 
similar  efforts.  Adjustments  are  made  to  the  contractor's 
proposal  reflecting  those  changes  the  SPAWAR  evaluators 
believe  necessary  to  make  the  proposal  more  realistic.  The 
adjusted  proposal  then  becomes  the  SPAWAR  estimate.  The 
goal  of  the  SPAWAR  evaluation  is  to  determine  whether  the 
contractor  has  the  capability  to  perform  the  required 
effort  and  what  the  cost  of  this  effort  will  be.  [Ref.  10] 
To  score  cost  realism,  the  contractor's  cost 
proposal  and  the  SPAWAR  cost  estimate  are  plotted  on  the 
cost  realism  scoring  rule  table  established  for  the 
procurement,  and  a  score  is  awarded.  The  cost  realism 
score  becomes  a  part  of  the  overall  score  for  the  proposal, 
as  delineated  in  the  source  selection  plan,  and  the 
proposal  with  the  highest  overall  score  is  awarded  the 
contract.  [Ref.  10] 

3 .  The  SPAWAR  Cost  Realism  Scoring  Rule 

As  previously  pointed  out,  the  difference  between 
the  contractor's  cost  proposal  and  the  SPAWAR  cost  estimate 
decides  the  cost  realism  score  as  determined  by  the  cost 
realism  scoring  rule.  The  principle  surrounding  the  cost 
realism  scoring  rule  is  very  simple,  as  demonstrated  in 
Figure  4.1.  [Ref.  10] 
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The  SPAWAR  Cost  Realism  Scorin 


Source:  Adapted  from  [Ref.  10] 

Figure  4.1 
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The  SPAWAR  cost  estimate  (S)  determines  the 
position  on  the  graph  where  the  maximum  score  can  be 


earned.  At  predetermined  distances  from  the  SPAWAR  cost 
estimate  are  the  points  B  and  B'  where  the  minimum  score 
will  be  earned.  When  the  contractor's  cost  proposal  (C)  is 
plotted,  the  score  associated  with  the  difference  (S-C)  is 
determined  by  the  line  BAB’.  The  closer  a  contractor's 
cost  proposal  comes  to  equalling  the  SPAWAR  cost  estimate, 
the  more  the  score  approaches  the  maximum.  The  reverse  is 
also  true.  As  the  contractor's  cost  proposal  moves  away 
from  the  SPAWAR  cost  estimate,  the  score  approaches  the 
minimum . 

The  shape  of  the  line  BAB'  associated  with  the  cost 
realism  scoring  rule  is  dependent  upon  the  particular 
procurement  under  consideration  and  the  assumptions 
associated  with  it.  Figure  4.2  represents  the  more  common 
shapes  used  consisting  of  the  triangle,  trapezoid,  stepped 
and  curve.  [Ref.  10] 

The  triangle  is  a  simple  scoring  rule  to  determine 
and  plot.  Between  B  or  B '  and  S  the  score  for  C  is 
proportional  to  the  difference  between  S  and  C. 

The  trapezoid  scoring  rule  allows  a  small  cost 
range  to  define  the  SPAWAR  cost  estimate  where  different 
cost  proposals  can  earn  the  maximum  score.  This  scoring 
rule  recognizes  the  uncertainty  which  can  surround  the 
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Common  Cost  Realism  Scoring  Rule  Shapes 
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Source:  Adapted  from  [Ref.  10] 


Figure  4.2 


development  of  the  government's  cost  estimate  and  allows 
credit  for  this  uncertainty. 

The  stepped  scoring  rule  takes  the  trapezoid  one 
step  further  by  assigning  the  same  score  to  a  range.  The 
problems  associated  with  this  rule  are  twofold.  First,  the 
stepped  scoring  rule  could  assign  quite  different  scores  to 
two  contractors  whose  cost  proposals  have  similar  degrees 
of  cost  realism  (i.e.  the  two  scores  straddle  a  step 
break).  Second,  the  stepped  scoring  rule  could  assign  the 
same  score  to  two  contractors  whose  cost  proposals  have 
quite  different  degrees  of  cost  realism  (i.e.  the  two 
scores  are  at  opposite  ends  of  a  step  plateau). 

The  use  of  the  curve  scoring  rule,  such  as  the 
normal  bell  shaped  curve,  provides  an  approximation  to  the 
triangle  and  trapezoid  scoring  rules. 

As  previously  stated,  the  type  and  shape  of  the 
cost  realism  scoring  rule  utilized  in  proposal  evaluation 
is  largely  determined  by  the  nature  of  the  procurement. 

The  scoring  spread  between  the  maximum  score  (S)  and  the 
minimum  score  (B  or  B')  is  usually  around  20  percent.  The 
belief  is  that  any  cost  proposal  that  is  more  than  20 
percent  away  from  a  carefully  prepared  SPAWAR  estimate 
raises  fundamental  questions  about  the  contractor's 
understanding  of  the  contract  requirements  and  hence  the 
cost  realism  of  the  proposal  [Ref.  10].  Of  course,  this  is 


not  to  say  that  a  scoring  spread  in  excess  of  twenty 
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Skewed  Cost  Realism  Scoring  Rule  Shapes 
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Mirce:  Adapted  from  [Ref.  10] 


Figure  4.3 


percent  cannot  be  established  if  the  nature  of  the 


procurement  requires  it. 

Another  item  considered  while  determining  the  type 
and  shape  of  the  cost  realism  scoring  rule  is  the 
distribution  of  the  scoring  spread  around  the  SPAWAR  cost 
estimate.  An  even,  symmetrical  distribution  is  normally 
provided,  but  there  can  be  procurements  where  the 
distribution  is  skewed  as  demonstrated  in  Figure  4.3. 

[Ref.  10] 


F .  SUMMARY 

The  cost  realism  approaches  taken  by  the  three  Navy 
systems  commands  appear  very  similar.  First,  each 
carefully  evaluates  contractor  cost  proposals  for 
completeness,  adquacy  and  supportability .  Then,  government 
cost  estimates  are  developed  using  the  contractor's 
proposal  as  a  basis.  Other  data  as  considered  desireable 
and  necessary  may  be  included.  Finally,  the  contractor's 
cost  proposal  is  compared  to  the  government's  cost  estimate 
to  determine  the  cost  realism  of  the  contractor's  cost 
proposal.  It  is  in  the  procedures  for  scoring  cost  realism 
that  the  three  systems  commands  differ  the  most,  with  only 
SPAWAR  having  developed  a  rather  formal  and  involved 
procedure.  The  end  results  for  the  three  systems  commands 
is  the  same,  however,  with  the  cost  realism  score  being 
included  as  a  part  of  the  final  score  determination  of  a 
contractor ' s  proposal . 


V.  CONCLUSIONS  AND  RECOMMENDATIONS 


ANSWERS  TO  THE  RESEARCH  QUESTIONS 


Can  the  cost  realism  concept  be  used  to  improve  the 


source  selection  decision  makin 


Cost  realism  requires  a  source  selection  decision  to 
consider  the  most  likely  cost  of  a  contractor's  effort. 

The  emphasis  given  to  budgetary  control  issues  makes  it 
important  that  a  source  selection  decision  consider  the 
most  likely  final  cost  of  a  procurement  before  making  an 
award.  With  cost  realism,  the  procurement  program  manager 
has  a  good  prediction  of  what  the  budgetary  requirements 
for  the  program  will  be. 

2.  What  is  cost  realism? 

Cost  realism  is  the  methodology  by  which  a  contractor's 

cost  proposal  is  evaluated  during  the  source  selection 
process  to  determine,  with  some  degree  of  accuracy  and 
fairness,  whether  or  not  it  is  a  realistic  estimate  of  the 
actual  costs,  assuming  no  changes  to  the  government's 
requirements  or  the  contractor’s  elements  of  cost  and/or 
technical  approach. 

3.  At  what  points  and  under  what  circumstances  may  cost 


realism  become  an  issue  in  the  source  selection 


First,  during  the  development  of  the  source  selection 
plan  when  planning  for  cost  realism  is  first  considered. 


The  cost  realism  methodology  to  be  employed  must  be  capable 
of  giving  adequate  consideration  to  the  various  risks 
associated  with  the  procurement  while  providing  a  realistic 


estimate  of  the  procurement's  actual  costs.  Second,  how 
cost  realism  will  be  weighted  and  scored  in  the  source 
selection  plan  may  become  an  issue  dependent  upon  the  other 
requirements  of  the  procurement  and  the  capabilities  of  the 
potential  offerors.  As  competing  proposals  are  evaluated 
and  considered  essentially  equal  in  the  other  evaluation 
factors,  (i.e.  technical,  management,  schedule,  cost),  cost 
realism  may  become  the  determining  factor.  An  inadequate 
cost  realism  analysis  could  result  in  award  to  an  offeror 
whose  proposal  is  not  the  most  advantageous  to  the 
government  and  the  protest  of  that  award  by  the 
unsuccessful  offeror(s). 

4 .  How  can  cost  realism  best  be  utilized  in  the  source 
selection  process? 

Cost  realism  can  best  be  utilized  as  an  evaluation 
factor  by  the  SSA  in  making  the  source  selection  decision. 
While  performing  a  cost  realism  analysis  the  contractor's 
proposal  receives  a  detailed  evaluation  that  provides  an 
estimate  of  the  most  likely  cost  of  the  contractor's 
proposal.  The  cost  realism  score  provides  an  invaluable 
method  to  compare  different  proposals  and  also  provides  a 
solid  basis  upon  which  to  defend  a  source  selection 
decision  if  that  should  become  necessary. 


B.  CONCLUSIONS 


The  following  conclusions  were  developed  as  a  result  of 
this  research  effort. 

Cost  realism  analysis  when  adequately  performed  may 
assist  the  Source  Selection  Authority  in  making  an 
informed  judgement  source  selection  decision. 

As  discussed,  cost  realism  can  be  one  of  many 

evaluation  factors  considered  by  the  SSA  in  making  a  source 

selection  decision.  As  competing  proposals  become  more 

equal  in  non-cost  related  evaluation  factors  (i.e. 

technical,  management,  schedule),  the  cost  related 

evaluation  factors  increase  in  importance  as  potentially 

deciding  factors.  The  necessity  for  an  informed  judgement 

with  regard  to  the  cost  related  evaluation  factors 

increases  the  probability  that  a  source  selection  decision 

will  be  supportable  and  defensible. 

An  adequate  cost  realism  analysis  requires  a  thorough  and 
complete  review  of  all  aspects  of  a  contractor's  proposal 
to  develop  a  costing  baseline  for  the  government 
estimate . 

It  is  in  comparison  against  the  evaluated  government 
estimate  that  the  cost  realism  of  a  contractor's  proposal 
is  determined.  The  government  estimate  is  developed 
through  a  careful  and  complete  evaluation  of  the  elements 


of  cost  (material,  labor,  overhead),  which  make  up  the 


contractor’s  cost  proposal. 


I 

I 


N 


Cost  realism  is  an  excellent  forecaster  of  what  a  program 
will  most  likely  cost;  however,  cost  realism  is  not  a 
guarantee  of  what  the  program's  final  cost  will  be . 

The  government  estimate  of  a  contractor's  most  likely 
cost  is  still  just  an  estimate.  It  is  subject  to  the  risks 
inherent  to  most  cost  reimbursable  procurements:  schedule, 
risk,  design  risk  and  production  risk.  If  all  risk  could 
be  eliminated  from  a  program,  there  would  be  no  need  for  a 
cost  reimbursable  contract  award.  Bids  could  be  taken  from 
potential  offerors,  and  the  award  would  be  made  to  the 
lowest  cost  bidder.  Since  risk  exists,  there  is  the 
distinct  possibility  that  a  program  will  experience  cost 
growth  changes  to  the  estimated  final  cost. 


C.  RECOMMENDATIONS 

The  following  recommendations  are  relevant  from  this 
research  effort. 

A  standard  methodology  for  determining  the  cost  realism 
of  a  contractor's  proposal  should  be  established  and 
required  for  all  cost  reimbursable  contract  awards. 

The  FAR  and  service  implementing  directives  and 
instructions  should  be  revised  to  give  specific  guidance  on 
determining  the  cost  realism  of  a  contractor's  proposal. 
Developing  standards  for  cost  realism  analysis  and 
requiring  a  cost  realism  determination  for  contractor 
proposals  will  ensure  that  source  selection  decisions  are 
informed  judgments. 
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Cost  realism  should  be  used  for  any  procurement  where  the 
contractor  is  required  to  provide  cost  and  pricing  data. 

Sole  source  negotiations  for  a  cost  reimbursable 
contract  could  use  cost  realism  analysis  to  assist  in 
determining  a  fair  and  reasonable  price  for  the 
procurement . 

D.  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

A  study  should  be  conducted  to  evaluate  the  optimum 
characteristics  of  cost  realism  scoring  rules  and  the 
differing  procurement  circumstances  under  which  their  use 
can  be  maximized. 

A  study  should  be  conducted  to  determine  how  cost 
realism  is  evaluated  in  other  government  agencies. 

A  study  should  be  conducted  to  review  the  effect  cost 


realism  has  on  how  contractors  develop  and  present 
competitive  cost  proposals. 
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